Coding of significance maps and transform coefficient blocks

ABSTRACT

A higher coding efficiency for coding a significance map indicating positions of significant transform coefficients within a transform coefficient block is achieved by the scan order by which the sequentially extracted syntax elements indicating, for associated positions within the transform coefficient block, as to whether at the respective position a significant or insignificant transform coefficient is situated, are sequentially associated to the positions of the transform coefficient block, among the positions of the transform coefficient block depends on the positions of the significant transform coefficients indicated by previously associated syntax elements. Alternatively, the first-type elements may be context-adaptively entropy decoded using contexts which are individually selected for each of the syntax elements dependent on a number of significant transform coefficients in a neighborhood of the respective syntax element, indicated as being significant by any of the preceding syntax elements.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation Application of U.S. patentapplication Ser. No. 15/972,372, filed May 7, 2018, which is aContinuation Application of U.S. patent application Ser. No. 15/200,300,filed Jul. 1, 2016, now U.S. Pat. No. 9,998,741, which is a Continuationof U.S. patent application Ser. No. 13/648,538, filed Oct. 10, 2012, nowU.S. Pat. No. 9,894,368, which is a Continuation of InternationalApplication No. PCT/EP2011/055644, filed Apr. 11, 2011, and additionallyclaims priority to European Patent Application No. EP 10159766.4, filedApr. 13, 2010 and International Patent Application No.PCT/EP2010/054822, filed Apr. 13, 2010, each of which is incorporatedherein by reference in entirety.

BACKGROUND OF THE INVENTION

The present application is directed to coding of significance mapsindicating positions of significant transform coefficients withintransform coefficient blocks and the coding of such transformcoefficient blocks. Such coding may, for example, be used in picture andvideo coding, for example.

In conventional video coding, the pictures of a video sequence areusually decomposed into blocks. The blocks or the color components ofthe blocks are predicted by either motion-compensated prediction orintra prediction. The blocks can have different sizes and can be eitherquadratic or rectangular. All samples of a block or a color component ofa block are predicted using the same set of prediction parameters, suchas reference indices (identifying a reference picture in the alreadycoded set of pictures), motion parameters (specifying a measure for themovement of a blocks between a reference picture and the currentpicture), parameters for specifying the interpolation filter, intraprediction modes, etc. The motion parameters can be represented bydisplacement vectors with a horizontal and vertical component or byhigher order motion parameters such as affine motion parametersconsisting of 6 components. It is also possible that more than one setof prediction parameters (such as reference indices and motionparameters) are associated with a single block. In that case, for eachset of prediction parameters, a single intermediate prediction signalfor the block or the color component of a block is generated, and thefinal prediction signal is built by a weighted sum of the intermediateprediction signals. The weighting parameters and potentially also aconstant offset (which is added to the weighted sum) can either be fixedfor a picture, or a reference picture, or a set of reference pictures,or they can be included in the set of prediction parameters for thecorresponding block. Similarly, still images are also often decomposedinto blocks, and the blocks are predicted by an intra prediction method(which can be a spatial intra prediction method or a simple intraprediction method that predicts the DC component of the block). In acomer case, the prediction signal can also be zero.

The difference between the original blocks or the color components ofthe original blocks and the corresponding prediction signals, alsoreferred to as the residual signal, is usually transformed andquantized. A two-dimensional transform is applied to the residual signaland the resulting transform coefficients are quantized. For thistransform coding, the blocks or the color components of the blocks, forwhich a particular set of prediction parameters has been used, can befurther split before applying the transform. The transform blocks can beequal to or smaller than the blocks that are used for prediction. It isalso possible that a transform block includes more than one of theblocks that are used for prediction. Different transform blocks in astill image or a picture of a video sequence can have different sizesand the transform blocks can represent quadratic or rectangular blocks.

The resulting quantized transform coefficients, also referred to astransform coefficient levels, are then transmitted using entropy codingtechniques. Therefore, a block of transform coefficients levels isusually mapped onto a vector (i.e., an ordered set) of transformcoefficient values using a scan, where different scans can be used fordifferent blocks. Often a zig-zag scan is used. For blocks that containonly samples of one field of an interlaced frame (these blocks can beblocks in coded fields or field blocks in coded frames), it is alsocommon to use a different scan specifically designed for field blocks. Acommonly used entropy coding algorithm for encoding the resultingordered sequence of transform coefficients is run-level coding. Usually,a large number of the transform coefficient levels is zero, and a set ofsuccessive transform coefficient levels that are equal to zero can beefficiently represented by coding the number of successive transformcoefficient levels that are equal to zero (the run). For the remaining(non-zero) transform coefficients, the actual level is coded. There arevarious alternatives of run-level codes. The run before a non-zerocoefficient and the level of the non-zero transform coefficient can becoded together using a single symbol or code word. Often, specialsymbols for the end-of-block, which is sent after the last non-zerotransform coefficient, are included. Or it is possible to first encodethe number of non-zero transform coefficient levels, and depending onthis number, the levels and runs are coded.

A somewhat different approach is used in the highly efficient CABACentropy coding in H.264. Here, the coding of transform coefficientlevels is split into three steps. In the first step, a binary syntaxelement coded_block_flag is transmitted for each transform block, whichsignals whether the transform block contains significant transformcoefficient levels (i.e., transform coefficients that are non-zero). Ifthis syntax element indicates that significant transform coefficientlevels are present, a binary-valued significance map is coded, whichspecifies which of the transform coefficient levels have non-zerovalues. And then, in a reverse scan order, the values of the non-zerotransform coefficient levels are coded. The significance map is coded asfollows. For each coefficient in the scan order, a binary syntax elementsignificant_coeff_flag is coded, which specifies whether thecorresponding transform coefficient level is not equal to zero. If thesignificant_coeff_flag bin is equal to one, i.e., if a non-zerotransform coefficient level exists at this scanning position, a furtherbinary syntax element last_significant_coeff_flag is coded. This binindicates if the current significant transform coefficient level is thelast significant transform coefficient level inside the block or iffurther significant transform coefficient levels follow in scanningorder. If last_significant_coeff_flag indicates that no furthersignificant transform coefficients follow, no further syntax elementsare coded for specifying the significance map for the block. In the nextstep, the values of the significant transform coefficient levels arecoded, whose locations inside the block are already determined by thesignificance map. The values of significant transform coefficient levelsare coded in reverse scanning order by using the following three syntaxelements. The binary syntax element coeff_abs_greater_one indicates ifthe absolute value of the significant transform coefficient level isgreater than one. If the binary syntax element coeff_abs_greater_oneindicates that the absolute value is greater than one, a further syntaxelement coeff_abs_level_minus_one is sent, which specifies the absolutevalue of the transform coefficient level minus one. Finally, the binarysyntax element coeff_sign_flag, which specifies the sign of thetransform coefficient value, is coded for each significant transformcoefficient level. It should be noted again that the syntax elementsthat are related to the significance map are coded in scanning order,whereas the syntax elements that are related to the actual values of thetransform coefficients levels are coded in reverse scanning orderallowing the usage of more suitable context models.

In the CABAC entropy coding in H.264, all syntax elements for thetransform coefficient levels are coded using a binary probabilitymodelling. The non-binary syntax element coeff_abs_level_minus_one isfirst binarized, i.e., it is mapped onto a sequence of binary decisions(bins), and these bins are sequentially coded. The binary syntaxelements significant_coeff_flag, last_significant_coeff_flag,coeff_abs_greater_one, and coeff_sign_flag are directly coded. Eachcoded bin (including the binary syntax elements) is associated with acontext. A context represents a probability model for a class of codedbins. A measure related to the probability for one of the two possiblebin values is estimated for each context based on the values of the binsthat have been already coded with the corresponding context. For severalbins related to the transform coding, the context that is used forcoding is selected based on already transmitted syntax elements or basedon the position inside a block.

The significance map specifies information about the significance(transform coefficient level is different from zero) for the scanpositions. In the CABAC entropy coding of H.264, for a block size of4×4, a separate context is used for each scan position for coding thebinary syntax elements significant_coeff_flag and thelast_significant_coeff_flag, where different contexts are used for thesignificant_coeff_flag and the last_significant_coeff_flag of a scanposition. For 8×8 blocks, the same context model is used for foursuccessive scan positions, resulting in 16 context models for thesignificant_coeff_flag and additional 16 context models for thelast_significant_coeff_flag.

This method of context modelling for the significant_coeff_flag and thelast_significant_coeff_flag has some disadvantages for large blocksizes. On the one hand side, if each scan position is associated with aseparate context model, the number of context models does significantlyincrease when blocks greater than 8×8 are coded. Such an increasednumber of context models results in a slow adaptation of the probabilityestimates and usually an inaccuracy of the probability estimates, whereboth aspects have a negative impact on the coding efficiency. On theother hand, the assignment of a context model to a number of successivescan positions (as done for 8×8 blocks in H.264) is also not optimal forlarger block sizes, since the non-zero transform coefficients areusually concentrated in particular regions of a transform block (theregions are dependent on the main structures inside the correspondingblocks of the residual signal).

After coding the significance map, the block is processed in reversescan order. If a scan position is significant, i.e., the coefficient isdifferent from zero, the binary syntax element coeff_abs_greater_one istransmitted. Initially, the second context model of the correspondingcontext model set is selected for the coeff_abs_greater_one syntaxelement. If the coded value of any coeff_abs_greater_one syntax elementinside the block is equal to one (i.e., the absolute coefficient isgreater than 2), the context modelling switches back to the firstcontext model of the set and uses this context model up to the end ofthe block. Otherwise (all coded values of coeff_abs_greater_one insidethe block are zero and the corresponding absolute coefficient levels areequal to one), the context model is chosen depending on the number ofthe coeff_abs_greater_one syntax elements equal to zero that havealready been coded/decoded in the reverse scan of the considered block.The context model selection for the syntax element coeff_abs_greater_onecan be summarized by the following equation, where the current contextmodel index C_(t)+1 is selected based on the previous context modelindex C_(t) and the value of the previously coded syntax elementcoeff_abs_greater_one, which is represented by bins in the equation. Forthe first syntax element coeff_abs_greater_one inside a block, thecontext model index is set equal to C_(t)=1.

${C_{t + 1}\left( {C_{t},{bin}_{t}} \right)} = \left\{ \begin{matrix}{0,} & {{{for}\mspace{20mu} {bin}_{t}} = 1} \\{\min \left( {{C_{t} + 1},4} \right)} & {{{for}\mspace{14mu} {bin}_{t}} = 0}\end{matrix} \right.$

The second syntax element for coding the absolute transform coefficientlevels, coeff_abs_level_minus_one is only coded, when thecoeff_abs_greater_one syntax element for the same scan position is equalto one. The non-binary syntax element coeff_abs_level_minus_one isbinarized into a sequence of bins and for the first bin of thisbinarization; a context model index is selected as described in thefollowing. The remaining bins of the binarization are coded with fixedcontexts. The context for the first bin of the binarization is selectedas follows. For the first coeff_abs_level_minus_one syntax element, thefirst context model of the set of context models for the first bin ofthe coeff_abs_level_minus_one syntax element is selected, thecorresponding context model index is set equal to C_(t)=0. For eachfurther first bin of the coeff_abs_level_minus_one syntax element, thecontext modelling switches to the next context model in the set, wherethe number of context models in set is limited to 5. The context modelselection can be expressed by the following formula, where the currentcontext model index C_(t+1) is selected based on the previous contextmodel index C_(t). As mentioned above, for the first syntax elementcoeff_abs_level_minus_one inside a block, the context model index is setequal to C_(t)=0. Note, that different sets of context models are usedfor the syntax elements coeff_abs_greater_one andcoeff_abs_level_minus_one.

C _(t+1)(C _(t))=min(C _(t)+1,4)

For large blocks, this method has some disadvantages. The selection ofthe first context model for coeff_abs_greater_one (which is used if avalue of coeff_abs_greater_one equal to 1 has been coded for the blocks)is usually done too early and the last context model forcoeff_abs_level_minus_one is reached too fast because the number ofsignificant coefficients is larger than in small blocks. So, most binsof coeff_abs_greater_one and coeff_abs_level_minus_one are coded with asingle context model. But these bins usually have differentprobabilities, and hence the usage of a single context model for a largenumber of bins has a negative impact on the coding efficiency.

Although, in general, large blocks increase the computational overheadfor performing the spectral decomposing transform, the ability toeffectively code both small and large blocks would enable theachievement of better coding efficiency in coding sample arrays such aspictures or sample arrays representing other spatially sampledinformation signals such as depth maps or the like. The reason for thisis the dependency between spatial and spectral resolution whentransforming a sample array within blocks: the larger the blocks thehigher the spectral resolution of the transform is. Generally, it wouldbe favorable to be able to locally apply the individual transform on asample array such that within the area of such an individual transform,the spectral composition of the sample array does not vary to a greatextent. To small blocks guarantee that the content within the blocks isrelatively consistent. On the other hand, if the blocks are too small,the spectral resolution is low, and the ratio between non-significantand significant transform coefficients gets lower.

Thus, it would be favorable to have a coding scheme which enables anefficient coding for transform coefficient blocks, even when they arelarge, and their significance maps.

SUMMARY

According to an embodiment, an apparatus for decoding a significance mapindicating positions of significant transform coefficients within atransform coefficient block from a data stream may have a decoderconfigured to sequentially extract first-type syntax elements from thedata stream, the first-type syntax elements indicating, for associatedpositions within the transform coefficient block, at least, as towhether at the respective position a significant or insignificanttransform coefficient is situated; and an associator configured tosequentially associate the sequentially extracted first-type syntaxelements to the positions of the transform coefficient block in a scanorder among the positions of the transform coefficient block, whichdepends on the positions of the significant transform coefficientsindicated by previously extracted and associated first-type syntaxelements.

According to another embodiment, an apparatus for decoding asignificance map indicating positions of significant transformcoefficients within a transform coefficient block from a data stream mayhave an decoder configured to extract a significance map indicatingpositions of significant transform coefficients within the transformcoefficient block, and then the values of the significant transformcoefficients within the transform coefficient block from a data stream,with, in extracting the significance map, sequentially extractingfirst-type syntax elements from the data stream by context-adaptiveentropy decoding, the first-type syntax elements indicating, forassociated positions within the transform coefficient block as towhether at the respective position a significant or insignificanttransform coefficient is situated; and an associator configured tosequentially associate the sequentially extracted first-type syntaxelements to the positions of the transform coefficient block in apredetermined scan order among the positions of the transformcoefficient block, wherein the decoder is configured to use, incontext-adaptively entropy decoding the first-type syntax elements,contexts which are individually selected for each of the first-typesyntax elements depending on a number of positions at which according tothe previously extracted and associated first-type syntax elementssignificant transform coefficients are situated, in a neighborhood ofthe position with which a current first-type syntax element isassociated.

According to another embodiment, an apparatus for decoding a transformcoefficient block may have a decoder configured to extract asignificance map indicating positions of significant transformcoefficients within the transform coefficient block, and then the valuesof the significant transform coefficients within the transformcoefficient block from a data stream, with, in extracting the values ofthe significant transform coefficients, sequentially extracting thevalues by context-adaptive entropy decoding; and an associatorconfigured to sequentially associate the sequentially extracted valueswith the positions of the significant transform coefficients in apredetermined coefficient scan order among the positions of thetransform coefficient block, according to which the transformcoefficient block is scanned in sub-blocks of the transform coefficientblock using a sub-block scan order with, subsidiary, scanning thepositions of the transform coefficients within the sub-blocks in aposition sub-scan order, wherein the decoder is configured to use, insequentially context-adapted entropy decoding the values of thesignificant transform coefficient values, a selected set of a number ofcontexts from a plurality of sets of a number of contexts, the selectionof the selected set being performed for each sub-block depending on thevalues of the transform coefficients within a sub-block of the transformcoefficient block, already having been traversed in the sub-block scanorder, or the values of the transform coefficients of a co-locatedsub-block in an equally sized previously decoded transform coefficientblock.

According to another embodiment, a transform-based decoder configured todecode a transform coefficient block using an apparatus decoding asignificance map indicating positions of significant transformcoefficients within a transform coefficient block from a data stream mayhave an decoder configured to sequentially extract first-type syntaxelements from the data stream, the first-type syntax elementsindicating, for associated positions within the transform coefficientblock, at least, as to whether at the respective position a significantor insignificant transform coefficient is situated; and an associatorconfigured to sequentially associate the sequentially extractedfirst-type syntax elements to the positions of the transform coefficientblock in a scan order among the positions of the transform coefficientblock, which depends on the positions of the significant transformcoefficients indicated by previously extracted and associated first-typesyntax elements, and to perform a transform from spectral domain tospatial domain to the transform coefficient block.

According to another embodiment, a transform-based decoder configured todecode a transform coefficient block using an apparatus for decoding asignificance map indicating positions of significant transformcoefficients within a transform coefficient block from a data stream mayhave an decoder configured to extract a significance map indicatingpositions of significant transform coefficients within the transformcoefficient block, and then the values of the significant transformcoefficients within the transform coefficient block from a data stream,with, in extracting the significance map, sequentially extractingfirst-type syntax elements from the data stream by context-adaptiveentropy decoding, the first-type syntax elements indicating, forassociated positions within the transform coefficient block as towhether at the respective position a significant or insignificanttransform coefficient is situated; and an associator configured tosequentially associate the sequentially extracted first-type syntaxelements to the positions of the transform coefficient block in apredetermined scan order among the positions of the transformcoefficient block, wherein the decoder is configured to use, incontext-adaptively entropy decoding the first-type syntax elements,contexts which are individually selected for each of the first-typesyntax elements depending on a number of positions at which according tothe previously extracted and associated first-type syntax elementssignificant transform coefficients are situated, in a neighborhood ofthe position with which a current first-type syntax element isassociated, and to perform a transform from spectral domain to spatialdomain to the transform coefficient block.

According to another embodiment, a predictive decoder may have atransform-based decoder configured to decode a transform coefficientblock using an apparatus for decoding a significance map indicatingpositions of significant transform coefficients within a transformcoefficient block from a data stream, wherein the apparatus may have andecoder configured to sequentially extract first-type syntax elementsfrom the data stream, the first-type syntax elements indicating, forassociated positions within the transform coefficient block, at least,as to whether at the respective position a significant or insignificanttransform coefficient is situated; and an associator configured tosequentially associate the sequentially extracted first-type syntaxelements to the positions of the transform coefficient block in a scanorder among the positions of the transform coefficient block, whichdepends on the positions of the significant transform coefficientsindicated by previously extracted and associated first-type syntaxelements, and to perform a transform from spectral domain to spatialdomain to the transform coefficient block to obtain a residual block; apredictor configured to provide a prediction for a block of an array ofinformation samples representing an spatially sampled informationsignal; and a combiner configured to combine the prediction of the blockand the residual block to reconstruct the array of information samples.

According to another embodiment, a predictive decoder may have atransform-based decoder configured to decode a transform coefficientblock using an apparatus for decoding a significance map indicatingpositions of significant transform coefficients within a transformcoefficient block from a data stream which may have an decoderconfigured to extract a significance map indicating positions ofsignificant transform coefficients within the transform coefficientblock, and then the values of the significant transform coefficientswithin the transform coefficient block from a data stream, with, inextracting the significance map, sequentially extracting first-typesyntax elements from the data stream by context-adaptive entropydecoding, the first-type syntax elements indicating, for associatedpositions within the transform coefficient block as to whether at therespective position a significant or insignificant transform coefficientis situated; and an associator configured to sequentially associate thesequentially extracted first-type syntax elements to the positions ofthe transform coefficient block in a predetermined scan order among thepositions of the transform coefficient block, wherein the decoder isconfigured to use, in context-adaptively entropy decoding the first-typesyntax elements, contexts which are individually selected for each ofthe first-type syntax elements depending on a number of positions atwhich according to the previously extracted and associated first-typesyntax elements significant transform coefficients are situated, in aneighborhood of the position with which a current first-type syntaxelement is associated, and to perform a transform from spectral domainto spatial domain to the transform coefficient block to obtain aresidual block; a predictor configured to provide a prediction for ablock of an array of information samples representing an spatiallysampled information signal; and a combiner configured to combine theprediction of the block and the residual block to reconstruct the arrayof information samples.

Another embodiment may have an apparatus for encoding a significance mapindicating positions of significant transform coefficients within atransform coefficient block into a data stream, the apparatus beingconfigured to sequentially code first-type syntax elements into the datastream by entropy encoding, the first-type syntax elements indicating,for associated positions within the transform coefficient block, atleast, as to whether at the respective position a significant orinsignificant transform coefficient is situated, wherein the apparatusis further configured to the first-type syntax elements into the datastream at a scan order among the positions of the transform coefficientblock, which depends on the positions of the significant transformcoefficients indicated by previously coded first-type syntax elements.

Another embodiment may have an apparatus for encoding a significance mapindicating positions of significant transform coefficients within atransform coefficient block into a data stream, the apparatus beingconfigured to code a significance map indicating positions ofsignificant transform coefficients within the transform coefficientblock, and then the values of the significant transform coefficientswithin the transform coefficient block into the data stream, with, incoding the significance map, sequentially coding first-type syntaxelements into the data stream by context-adaptive entropy encoding, thefirst-type syntax elements indicating, for associated positions withinthe transform coefficient block as to whether at the respective positiona significant or insignificant transform coefficient is situated,wherein the apparatus is further configured to sequentially code thefirst-type syntax elements into the data stream in a predetermined scanorder among the positions of the transform coefficient block, whereinthe apparatus is configured to use, in context-adaptively entropyencoding each of the first-type syntax elements, contexts which areindividually selected for the first-type syntax elements depending on anumber of positions at which significant transform coefficients aresituated and with which the previously coded first-type syntax elementsare associated, in a neighborhood of the position with which a currentfirst-type syntax element is associated.

Another embodiment may have an apparatus for encoding a transformcoefficient block, configured to code a significance map indicatingpositions of significant transform coefficients within the transformcoefficient block, and then the values of the significant transformcoefficients within the transform coefficient block into a data stream,with, in extracting the values of the significant transformcoefficients, sequentially coding the values by context-adaptive entropyencoding, wherein the apparatus is configured to code the values intothe data stream in a predetermined coefficient scan order among thepositions of the transform coefficient block, according to which thetransform coefficient block is scanned in sub-blocks of the transformcoefficient block using a sub-block scan order with, subsidiary,scanning the positions of the transform coefficients within thesub-blocks in a position sub-scan order, wherein the apparatus isfurther configured to use, in sequentially context-adapted entropyencoding the values of the significant transform coefficient values, aselected set of a number of contexts from a plurality of sets of anumber of contexts, the selection of the selected set being performedfor each sub-block depending on the values of the transform coefficientswithin a sub-block of the transform coefficient block, already havingbeen traversed in the sub-block scan order, or the values of thetransform coefficients of a co-located sub-block in an equally sizedpreviously encoded transform coefficient block.

According to another embodiment, a method for decoding a significancemap indicating positions of significant transform coefficients within atransform coefficient block from a data stream may have the steps ofsequentially extracting first-type syntax elements from the data stream,the first-type syntax elements indicating, for associated positionswithin the transform coefficient block, at least, as to whether at therespective position a significant or insignificant transform coefficientis situated; and sequentially associating the sequentially extractedfirst-type syntax elements to the positions of the transform coefficientblock in a scan order among the positions of the transform coefficientblock, which depends on the positions of the significant transformcoefficients indicated by previously extracted and associated first-typesyntax elements.

According to another embodiment, a method for decoding a significancemap indicating positions of significant transform coefficients within atransform coefficient block from a data stream may have the steps ofextracting a significance map indicating positions of significanttransform coefficients within the transform coefficient block, and thenthe values of the significant transform coefficients within thetransform coefficient block from a data stream, with, in extracting thesignificance map, sequentially extracting first-type syntax elementsfrom the data stream by context-adaptive entropy decoding, thefirst-type syntax elements indicating, for associated positions withinthe transform coefficient block as to whether at the respective positiona significant or insignificant transform coefficient is situated; andsequentially associating the sequentially extracted first-type syntaxelements to the positions of the transform coefficient block in apredetermined scan order among the positions of the transformcoefficient block, wherein, in context-adaptively entropy decoding thefirst-type syntax elements, contexts are used which are individuallyselected for each of the first-type syntax elements depending on anumber of positions at which according to the previously extracted andassociated first-type syntax elements significant transform coefficientsare situated, in a neighborhood of the position with which a currentfirst-type syntax element is associated.

According to another embodiment, a method for decoding a transformcoefficient block may have the steps of extracting a significance mapindicating positions of significant transform coefficients within thetransform coefficient block, and then the values of the significanttransform coefficients within the transform coefficient block from adata stream, with, in extracting the values of the significant transformcoefficients, sequentially extracting the values by context-adaptiveentropy decoding; and sequentially associating the sequentiallyextracted values with the positions of the significant transformcoefficients in a predetermined coefficient scan order among thepositions of the transform coefficient block, according to which thetransform coefficient block is scanned in sub-blocks of the transformcoefficient block using a sub-block scan order with, subsidiary,scanning the positions of the transform coefficients within thesub-blocks in a position sub-scan order, wherein, in sequentiallycontext-adapted entropy decoding the values of the significant transformcoefficient values, a selected set of a number of contexts from aplurality of sets of a number of contexts is used, the selection of theselected set being performed for each sub-block depending on the valuesof the transform coefficients within a sub-block of the transformcoefficient block, already having been traversed in the sub-block scanorder, or the values of the transform coefficients of a co-locatedsub-block in an equally sized previously decoded transform coefficientblock.

According to another embodiment, a method for encoding a significancemap indicating positions of significant transform coefficients within atransform coefficient block into a data stream may have the steps ofsequentially coding first-type syntax elements into the data stream byentropy encoding, the first-type syntax elements indicating, forassociated positions within the transform coefficient block, at least,as to whether at the respective position a significant or insignificanttransform coefficient is situated, with coding the first-type syntaxelements into the data stream at a scan order among the positions of thetransform coefficient block, which depends on the positions of thesignificant transform coefficients indicated by previously codedfirst-type syntax elements.

According to another embodiment, a method for encoding a significancemap indicating positions of significant transform coefficients within atransform coefficient block into a data stream may have the steps ofcoding a significance map indicating positions of significant transformcoefficients within the transform coefficient block, and then the valuesof the significant transform coefficients within the transformcoefficient block into the data stream, with, in coding the significancemap, sequentially coding first-type syntax elements into the data streamby context-adaptive entropy encoding, the first-type syntax elementsindicating, for associated positions within the transform coefficientblock as to whether at the respective position a significant orinsignificant transform coefficient is situated, wherein thesequentially coding the first-type syntax elements into the data streamis performed in a predetermined scan order among the positions of thetransform coefficient block, and in context-adaptively entropy encodingeach of the first-type syntax elements, contexts are used which areindividually selected for the first-type syntax elements depending on anumber of positions at which significant transform coefficients aresituated and with which the previously coded first-type syntax elementsare associated, in a neighborhood of the position with which a currentfirst-type syntax element is associated.

According to another embodiment, a method for encoding a transformcoefficient block may have the steps of coding a significance mapindicating positions of significant transform coefficients within thetransform coefficient block, and then the values of the significanttransform coefficients within the transform coefficient block into adata stream, with, in coding the values of the significant transformcoefficients, sequentially coding the values by context-adaptive entropyencoding, wherein the coding the values into the data stream isperformed in a predetermined coefficient scan order among the positionsof the transform coefficient block, according to which the transformcoefficient block is scanned in sub-blocks of the transform coefficientblock using a sub-block scan order with, subsidiary, scanning thepositions of the transform coefficients within the sub-blocks in aposition sub-scan order, wherein in sequentially context-adapted entropyencoding the values of the significant transform coefficient values, aselected set of a number of contexts from a plurality of sets of anumber of contexts is used, the selection of the selected set beingperformed for each sub-block depending on the values of the transformcoefficients within a sub-block of the transform coefficient block,already having been traversed in the sub-block scan order, or the valuesof the transform coefficients of a co-located sub-block in an equallysized previously encoded transform coefficient block.

Another embodiment may be a data stream having encoded therein asignificance map indicating positions of significant transformcoefficients within a transform coefficient block, wherein first-typesyntax elements are sequentially coded into the data stream by entropyencoding, the first-type syntax elements indicating, for associatedpositions within the transform coefficient block, at least, as towhether at the respective position a significant or insignificanttransform coefficient is situated, wherein the first-type syntaxelements are coded into the data stream at a scan order among thepositions of the transform coefficient block, which depends on thepositions of the significant transform coefficients indicated bypreviously coded first-type syntax elements.

Another embodiment may be a data stream having encoded thereinsignificance map indicating positions of significant transformcoefficients within a transform coefficient block, wherein asignificance map indicating positions of significant transformcoefficients within the transform coefficient block, followed by thevalues of the significant transform coefficients within the transformcoefficient block are coded into the data stream, wherein, within thesignificance map, the first-type syntax elements are sequentially codesinto the data stream by context-adaptive entropy encoding, thefirst-type syntax elements indicating, for associated positions withinthe transform coefficient block as to whether at the respective positiona significant or insignificant transform coefficient is situated,wherein the first-type syntax elements are sequentially coding into thedata stream in a predetermined scan order among the positions of thetransform coefficient block, and the first-type syntax elements arecontext-adaptively entropy encoded into the data stream using contextswhich are individually selected for the first-type syntax elementsdepending on a number of positions at which significant transformcoefficients are situated and with which the preceding first-type syntaxelements coded into the data stream are associated, in a neighborhood ofthe position with which a current first-type syntax element isassociated.

Another embodiment may be a data stream having encoded a coding of asignificance map indicating positions of significant transformcoefficients within the transform coefficient block, followed by thevalues of the significant transform coefficients within the transformcoefficient block, wherein the values of the significant transformcoefficients are sequentially coded into the data stream bycontext-adaptive entropy encoding in a predetermined coefficient scanorder among the positions of the transform coefficient block, accordingto which the transform coefficient block is scanned in sub-blocks of thetransform coefficient block using a sub-block scan order with,subsidiary, scanning the positions of the transform coefficients withinthe sub-blocks in a position sub-scan order, wherein the values of thesignificant transform coefficient values are sequentiallycontext-adapted entropy encoded into the data stream using a selectedset of a number of contexts from a plurality of sets of a number ofcontexts, the selection of the selected set being performed for eachsub-block depending on the values of the transform coefficients within asub-block of the transform coefficient block, already having beentraversed in the sub-block scan order, or the values of the transformcoefficients of a co-located sub-block in an equally sized previouslyencoded transform coefficient block.

Another embodiment may be a computer readable digital storage mediumhaving stored thereon a computer program having a program code forperforming, when running on a computer, a method for decoding asignificance map indicating positions of significant transformcoefficients within a transform coefficient block from a data streamhaving the steps of extracting a significance map indicating positionsof significant transform coefficients within the transform coefficientblock, and then the values of the significant transform coefficientswithin the transform coefficient block from a data stream, with, inextracting the significance map, sequentially extracting first-typesyntax elements from the data stream by context-adaptive entropydecoding, the first-type syntax elements indicating, for associatedpositions within the transform coefficient block as to whether at therespective position a significant or insignificant transform coefficientis situated; and sequentially associating the sequentially extractedfirst-type syntax elements to the positions of the transform coefficientblock in a predetermined scan order among the positions of the transformcoefficient block, wherein, in context-adaptively entropy decoding thefirst-type syntax elements, contexts are used which are individuallyselected for each of the first-type syntax elements depending on anumber of positions at which according to the previously extracted andassociated first-type syntax elements significant transform coefficientsare situated, in a neighborhood of the position with which a currentfirst-type syntax element is associated.

Another embodiment may be a computer readable digital storage mediumhaving stored thereon a computer program having a program code forperforming, when running on a computer, a method for decoding atransform coefficient block, which may have the steps of extracting asignificance map indicating positions of significant transformcoefficients within the transform coefficient block, and then the valuesof the significant transform coefficients within the transformcoefficient block from a data stream, with, in extracting the values ofthe significant transform coefficients, sequentially extracting thevalues by context-adaptive entropy decoding; and sequentiallyassociating the sequentially extracted values with the positions of thesignificant transform coefficients in a predetermined coefficient scanorder among the positions of the transform coefficient block, accordingto which the transform coefficient block is scanned in sub-blocks of thetransform coefficient block using a sub-block scan order with,subsidiary, scanning the positions of the transform coefficients withinthe sub-blocks in a position sub-scan order, wherein, in sequentiallycontext-adapted entropy decoding the values of the significant transformcoefficient values, a selected set of a number of contexts from aplurality of sets of a number of contexts is used, the selection of theselected set being performed for each sub-block depending on the valuesof the transform coefficients within a sub-block of the transformcoefficient block, already having been traversed in the sub-block scanorder, or the values of the transform coefficients of a co-locatedsub-block in an equally sized previously decoded transform coefficientblock.

Another embodiment may be a computer readable digital storage mediumhaving stored thereon a computer program having a program code forperforming, when running on a computer, a method for encoding asignificance map indicating positions of significant transformcoefficients within a transform coefficient block into a data stream,which may have the steps of sequentially coding first-type syntaxelements into the data stream by entropy encoding, the first-type syntaxelements indicating, for associated positions within the transformcoefficient block, at least, as to whether at the respective position asignificant or insignificant transform coefficient is situated, withcoding the first-type syntax elements into the data stream at a scanorder among the positions of the transform coefficient block, whichdepends on the positions of the significant transform coefficientsindicated by previously coded first-type syntax elements.

In accordance with a first aspect of the present application, anunderlying idea of the present application is that a higher codingefficiency for coding a significance map indicating positions ofsignificant transform coefficients within a transform coefficient blockmay be achieved if the scan order by which the sequentially extractedsyntax elements indicating, for associated positions within thetransform coefficient block, as to whether at the respective position asignificant or insignificant transform coefficient is situated, aresequentially associated to the positions of the transform coefficientblock, among the positions of the transform coefficient block depends onthe positions of the significant transform coefficients indicated bypreviously associated syntax elements. In particular, the inventorsfound out that in typical sample array content such as picture, video ordepth map content, the significant transform coefficients mostly formclusters at a certain side of the transform coefficient blockcorresponding to either non-zero frequencies in the vertical and lowfrequencies in the horizontal direction or vice versa so that takinginto account the positions of significant transform coefficientsindicated by previously associated syntax elements enables to controlthe further cause of the scan such that the probability of reaching thelast significant transform coefficient within the transform coefficientblock earlier is increased relative to a procedure according to whichthe scan order is predetermined independent from the positions of thesignificant transform coefficients indicated by previously associatedsyntax elements so far. This is particularly true for larger blocks,although the just said is also true for small blocks.

In accordance with an embodiment of the present application, the entropydecoder is configured to extract from the data stream informationenabling to recognize as to whether a significant transform coefficientcurrently indicated by a currently associated syntax element is the lastsignificant transform coefficient independent from its exact positionwithin the transform coefficient block wherein the entropy decoder isconfigured to expect no further syntax element in case of the currentsyntax element relating to such last significant transform coefficient.This information may comprise the number of significant transformcoefficients within the block. Alternatively, second syntax elements areinterleaved with the first syntax elements, the second syntax elementsindicating, for associated positions at which a significant transformcoefficient is situated, as to whether same is the last transformcoefficient in the transform coefficient block or not.

In accordance with an embodiment, the associator adapts the scan orderdepending on the positions of the significant transform coefficientsindicated so far merely at predefined positions within the transformcoefficient block. For example, several sub-paths which traversemutually disjointed sub-sets of positions within the transformcoefficient block extend substantially diagonally from one pair of sidesof the transform coefficient block corresponding to minimum frequencyalong a first direction and highest frequency along the other direction,respectively, to an opposite pair of sides of the transform coefficientblock corresponding to zero frequency along the second direction andmaximum frequency along the first direction, respectively. In this casethe associator is configured to select the scan order such that thesub-paths are traversed in an order among the sub-paths where thedistance of the sub-paths to the DC position within the transformcoefficient block monotonically increases, each sub-path is traversedwithout interrupt along run direction, and for each sub-path thedirection along which the sub-path is traversed is selected by theassociator depending on the positions of the significant transformcoefficients having been traversed during the previous sub-paths. Bythis measure, the probability is increased that the last sub-path, wherethe last significant transform coefficient is situated, is traversed ina direction so that it is more probable that the last significanttransform coefficient lies within the first half of this last sub-paththan within the second half thereof, thereby enabling to reduce thenumber of syntax elements indicating as to whether at a respectiveposition a significant or insignificant transform coefficient issituated. The effect is especially valuable in case of large transformcoefficient blocks.

According to a further aspect of the present application, the presentapplication is based on the finding that a significance map indicatingpositions of significant transform coefficients within a transformcoefficient block may be coded more efficiently if the aforementionedsyntax elements indicating, for associated positions within thetransform coefficient block as to whether at the respective position asignificant or insignificant transform coefficient is situated, arecontext-adaptively entropy decoded using contexts which are individuallyselected for each of the syntax elements dependent on a number ofsignificant transform coefficients in a neighborhood of the respectivesyntax element, indicated as being significant by any of the precedingsyntax elements. In particular, the inventors found out that withincreasing size of the transform coefficient blocks, the significanttransform coefficients are somehow clustered at certain areas within thetransform coefficient block so that a context adaptation which is notonly sensitive to the number of significant transform coefficientshaving been traversed in the predetermined scan orders so far but alsotakes into account the neighborhood of the significant transformcoefficients results in a better adaptation of the context and thereforeincreases the coding efficiency of the entropy coding. Of course, bothof the above-outlined aspects may be combined in a favourable way.

Further, in accordance with an even further aspect of the presentapplication, the application is based on the finding that the codingefficiency for coding a transform coefficient block may be increasedwhen a significance map indicating positions of the significanttransform coefficients within the transform coefficient block precedesthe coding of the actual values of the significant transformcoefficients within the transform coefficient block and if thepredetermined scan order among the positions of the transformcoefficient block used to sequentially associate the sequence of valuesof the significant transform coefficients with the positions of thesignificant transform coefficients scans the transform coefficient blockin sub-blocks using a sub-block scan order among the sub-blocks with,subsidiary, scanning the positions of the transform coefficients withinthe sub-blocks in a coefficients scan order, and if a selected set of anumber of contexts from a plurality of sets of a number of context isused for sequentially context adaptively entropy decoding the values ofthe significant transform coefficient values, the selection of theselected set depending on the values of the transform coefficientswithin a sub-block of the transform coefficient block already havingbeen traversed in the sub-block scan order or the values of thetransform coefficients of a co-located sub-block in a previously decodedtransform coefficient block. This way the context adaptation is verywell suited to the above-outlined property of significant transformcoefficients being clustered at certain areas within a transformcoefficient block, especially when large transform coefficient blocksare considered. In other words, the values may be scanned in sub-blocks,and contexts selected based on sub-block statistics.

Again, even the latter aspect may be combined with any of the previouslyidentified aspects of the present application or with both aspects.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present application are described in the followingwith respect to the Figures among which

FIG. 1 shows a block diagram of an encoder according to an embodiment;

FIGS. 2a-2c schematically show different sub-divisions of a sample arraysuch as a picture into blocks;

FIG. 3 shows a block diagram of a decoder according to an embodiment;

FIG. 4 shows a block diagram of an encoder according to an embodiment ofthe present application in more detail;

FIG. 5 shows a block diagram of a decoder according to an embodiment ofthe present application in more detail;

FIG. 6 schematically illustrates a transform of a block from spatialdomain into spectral domain;

FIG. 7 shows a block diagram of an apparatus for decoding thesignificance map and the significant transform coefficients of atransform coefficient block in accordance with an embodiment;

FIG. 8 schematically illustrates a sub-partitioning of a scan order intosub-paths and their different traversal directions;

FIG. 9 schematically illustrates neighborhood definitions for certainscan positions within a transform block in accordance with anembodiment;

FIG. 10 schematically illustrates possible neighborhood definitions forsome scan positions within transform blocks lying at the border of atransform block;

FIG. 11 shows a possible scan of transform blocks in accordance with afurther embodiment of the present application.

DETAILED DESCRIPTION OF THE INVENTION

It is noted that during the description of the figures, elementsoccurring in several of these Figures are indicated with the samereference sign in each of these Figures and a repeated description ofthese elements as far as the functionality is concerned is avoided inorder to avoid unnecessary repetitions. Nevertheless, thefunctionalities and descriptions provided with respect to one figureshall also apply to other Figures unless the opposite is explicitlyindicated.

FIG. 1 shows an example for an encoder 10 in which aspects of thepresent application may be implemented. The encoder encodes an array ofinformation samples 20 into a data stream. The array of informationsamples may represent any kind of spatially sampled information signal.For example, the sample array 20 may be a still picture or a picture ofa video. Accordingly, the information samples may correspond tobrightness values, color values, luma values, chroma values or the like.However, the information samples may also be depth values in case of thesample array 20 being a depth map generated by, for example, a time oflight sensor or the like.

The encoder 10 is a block-based encoder. That is, encoder 10 encodes thesample array 20 into the data stream 30 in units of blocks 40. Theencoding in units of blocks 40 does not necessarily mean that encoder 10encodes these blocks 40 totally independent from each other. Rather,encoder 10 may use reconstructions of previously encoded blocks in orderto extrapolate or intra-predict remaining blocks, and may use thegranularity of the blocks for setting coding parameters, i.e. forsetting the way each sample array region corresponding to a respectiveblock is coded.

Further, encoder 10 is a transform coder. That is, encoder 10 encodesblocks 40 by using a transform in order to transfer the informationsamples within each block 40 from spatial domain into spectral domain. Atwo-dimensional transform such as a DCT of FFT or the like may be used.Advantageously, the blocks 40 are of quadratic shape or rectangularshape.

The sub-division of the sample array 20 into blocks 40 shown in FIG. 1merely serves for illustration purposes. FIG. 1 shows the sample array20 as being sub-divided into a regular two-dimensional arrangement ofquadratic or rectangular blocks 40 which abut to each other in anon-overlapping manner. The size of the blocks 40 may be predetermined.That is, encoder 10 may not transfer an information on the block size ofblocks 40 within the data stream 30 to the decoding side. For example,the decoder may expect the predetermined block size.

However, several alternatives are possible. For example, the blocks mayoverlap each other. The overlapping may, however, be restricted to suchan extent that each block has a portion not overlapped by anyneighbouring block, or such that each sample of the blocks is overlappedby, at the maximum, one block among the neighbouring blocks arranged injuxtaposition to the current block along a predetermined direction. Thatlatter would mean that the left and right hand neighbor blocks mayoverlap the current block so as to fully cover the current block butthey may not overlay each other, and the same applies for the neighborsin vertical and diagonal direction.

As a further alternative, the sub-division of sample array 20 intoblocks 40 may be adapted to the content of the sample array 20 by theencoder 10 with the sub-division information on the sub-division usedbeing transferred to the decoder side via bitstream 30.

FIGS. 2a to 2c show different examples for a sub-division of a samplearray 20 into blocks 40. FIG. 2a shows a quadtree-based sub-division ofa sample array 20 into blocks 40 of different sizes, with representativeblocks being indicated at 40 a, 40 b, 40 c and 40 d with increasingsize. In accordance with the sub-division of FIG. 2 a, the sample array20 is firstly divided into a regular two-dimensional arrangement of treeblocks 40 d which, in turn, have individual sub-division informationassociated therewith according to which a certain tree block 40 d may befurther sub-divided according to a quadtree structure or not. The treeblock to the left of block 40 d is exemplarily sub-divided into smallerblocks in accordance with a quadtree structure. The encoder 10 mayperform one two-dimensional transform for each of the blocks shown withsolid and dashed lines in FIG. 2 a. In other words, encoder 10 maytransform the array 20 in units of the block subdivision.

Instead of a quadtree-based sub-division a more general multi tree-basedsub-division may be used and the number of child nodes per hierarchylevel may differ between different hierarchy levels.

FIG. 2b shows another example for a sub-division. In accordance withFIG. 2 b, the sample array 20 is firstly divided into macroblocks 40 barranged in a regular two-dimensional arrangement in a non-overlappingmutually abutting manner wherein each macroblock 40 b has associatedtherewith sub-division information according to which a macroblock isnot sub-divided, or, if subdivided, sub-divided in a regulartwo-dimensional manner into equally-sized sub-blocks so as to achievedifferent sub-division granularities for different macroblocks. Theresult is a sub-division of the sample array 20 in differently-sizedblocks 40 with representatives of the different sizes being indicated at40 a, 40 b and 40 a′. As in FIG. 2 a, the encoder 10 performs atwo-dimensional transform on each of the blocks shown in FIG. 2b withthe solid and dashed lines. FIG. 2c will be discussed later.

FIG. 3 shows a decoder 50 being able to decode the data stream 30generated by encoder 10 to reconstruct a reconstructed version 60 of thesample array 20. Decoder 50 extracts from the data stream 30 thetransform coefficient block for each of the blocks 40 and reconstructsthe reconstructed version 60 by performing an inverse transform on eachof the transform coefficient blocks.

Encoder 10 and decoder 50 may be configured to perform entropyencoding/decoding in order to insert the information on the transformcoefficient blocks into, and extract this information from the datastream, respectively. Details in this regard are described later. Itshould be noted that the data stream 30 not necessarily comprisesinformation on transform coefficient blocks for all the blocks 40 of thesample array 20. Rather, as sub-set of blocks 40 may be coded into thebitstream 30 in another way. For example, encoder 10 may decide torefrain from inserting a transform coefficient block for a certain blockof blocks 40 with inserting into the bitstream 30 alternative codingparameters instead which enable the decoder 50 to predict or otherwisefill the respective block in the reconstructed version 60. For example,encoder 10 may perform a texture analysis in order to locate blockswithin sample array 20 which may be filled at the decoder side bydecoder by way of texture synthesis and indicate this within thebitstream accordingly.

As discussed in the following Figures, the transform coefficient blocksnot necessarily represent a spectral domain representation of theoriginal information samples of a respective block 40 of the samplearray 20. Rather, such a transform coefficient block may represent aspectral domain representation of a prediction residual of therespective block 40. FIG. 4 shows an embodiment for such an encoder. Theencoder of FIG. 4 comprises a transform stage 100, an entropy coder 102,an inverse transform stage 104, a predictor 106 and a subtractor 108 aswell as an adder 110. Subtractor 108, transform stage 100 and entropycoder 102 are serially connected in the order mentioned between an input112 and an output 114 of the encoder of FIG. 4. The inverse transformstage 104, adder 110 and predictor 106 are connected in the ordermentioned between the output of transform stage 100 and the invertinginput of subtractor 108, with the output of predictor 106 also beingconnected to a further input of adder 110.

The coder of FIG. 4 is a predictive transform-based block coder. Thatis, the blocks of a sample array 20 entering input 112 are predictedfrom previously encoded and reconstructed portions of the same samplearray 20 or previously coded and reconstructed other sample arrays whichmay precede or succeed the current sample array 20 in time. Theprediction is performed by predictor 106. Subtractor 108 subtracts theprediction from such an original block and the transform stage 100performs a two-dimensional transformation on the prediction residuals.The two-dimensional transformation itself or a subsequent measure insidetransform stage 100 may lead to a quantization of the transformationcoefficients within the transform coefficient blocks. The quantizedtransform coefficient blocks are losslessly coded by, for example,entropy encoding within entropy encoder 102 with the resulting datastream being output at output 114. The inverse transform stage 104reconstructs the quantized residual and adder 110, in turn, combines thereconstructed residual with the corresponding prediction in order toobtain reconstructed information samples based on which predictor 106may predict the afore-mentioned currently encoded prediction blocks.Predictor 106 may use different prediction modes such as intraprediction modes and inter prediction modes in order to predict theblocks and the prediction parameters are forwarded to entropy encoder102 for insertion into the data stream.

That is, in accordance with the embodiment of FIG. 4, the transformcoefficient blocks represent a spectral representation of a residual ofthe sample array rather than actual information samples thereof.

It should be noted that several alternatives exist for the embodiment ofFIG. 4 with some of them having been described within the introductoryportion of the specification which description is incorporated into thedescription of FIG. 4 herewith. For example, the prediction generated bypredictor 106 may not be entropy encoded. Rather, the side informationmay be transferred to the decoding side by way of another coding scheme.

FIG. 5 shows a decoder able to decode a data stream generated by theencoder of FIG. 4. The decoder of FIG. 5 comprises an entropy decoder150, an inverse transform stage 152, an adder 154 and a predictor 156.Entropy decoder 150, inverse transform stage 152, and adder 154 areserially connected between an input 158 and an output 160 of the decoderof FIG. 5 in the order mentioned. A further output of entropy decoder150 is connected to predictor 156 which, in turn, is connected betweenthe output of adder 154 and a further input thereof. The entropy decoder150 extracts, from the data stream entering the decoder of FIG. 5 atinput 158, the transform coefficient blocks wherein an inverse transformis applied to the transform coefficient blocks at stage 152 in order toobtain the residual signal. The residual signal is combined with aprediction from predictor 156 at adder 154 so as to obtain areconstructed block of the reconstructed version of the sample array atoutput 160. Based on the reconstructed versions, predictor 156 generatesthe predictions thereby rebuilding the predictions performed bypredictor 106 at the encoder side. In order to obtain the samepredictions as those used at the encoder side, predictor 156 uses theprediction parameters which the entropy decoder 150 also obtains fromthe data stream at input 158.

It should be noted that in the above-described embodiments, the spatialgranularity at which the prediction and the transformation of theresidual is performed, do not have to be equal to each other. This isshown in FIG. 2C. This figure shows a sub-division for the predictionblocks of the prediction granularity with solid lines and the residualgranularity with dashed lines. As can be seen, the subdivisions may beselected by the encoder independent from each other. To be more precise,the data stream syntax may allow for a definition of the residualsubdivision independent from the prediction subdivision. Alternatively,the residual subdivision may be an extension of the predictionsubdivision so that each residual block is either equal to or a propersubset of a prediction block. This is shown on FIG. 2a and FIG. 2 b, forexample, where again the prediction granularity is shown with solidlines and the residual granularity with dashed lines. This, in FIG. 2a-2 c, all blocks having a reference sign associated therewith would beresidual blocks for which one two-dimensional transform would beperformed while the greater solid line blocks encompassing the dashedline blocks 40 a, for example, would be prediction blocks for which aprediction parameter setting is performed individually.

The above embodiments have in common that a block of (residual ororiginal) samples is to be transformed at the encoder side into atransform coefficient block which, in turn, is to be inverse transformedinto a reconstructed block of samples at the decoder side. This isillustrated in FIG. 6. FIG. 6 shows a block of samples 200. In case ofFIG. 6, this block 200 is exemplarily quadratic and 4×4 samples 202 insize. The samples 202 are regularly arranged along a horizontaldirection x and vertical direction y. By the above-mentionedtwo-dimensional transform T, block 200 is transformed into spectraldomain, namely into a block 204 of transform coefficients 206, thetransform block 204 being of the same size as block 200. That is,transform block 204 has as many transform coefficients 206 as block 200has samples, in both horizontal direction and vertical direction.However, as transform T is a spectral transformation, the positions ofthe transform coefficients 206 within transform block 204 do notcorrespond to spatial positions but rather to spectral components of thecontent of block 200. In particular, the horizontal axis of transformblock 204 corresponds to an axis along which the spectral frequency inthe horizontal direction monotonically increases while the vertical axiscorresponds to an axis along which the spatial frequency in the verticaldirection monotonically increases wherein the DC component transformcoefficient is positioned in a corner—here exemplarily the top leftcorner—of block 204 so that at the bottom right-hand corner, thetransform coefficient 206 corresponding to the highest frequency in bothhorizontal and vertical direction is positioned. Neglecting the spatialdirection, the spatial frequency to which a certain transformcoefficient 206 belongs, generally increases from the top left corner tothe bottom right-hand corner. By an inverse transform T⁻¹, the transformblock 204 is re-transferred from spectral domain to spatial domain, soas to re-obtain a copy 208 of block 200. In case no quantization/losshas been introduced during the transformation, the reconstruction wouldbe perfect.

As already noted above, it may be seen from FIG. 6 that greater blocksizes of block 200 increase the spectral resolution of the resultingspectral representation 204. On the other hand, quantization noise tendsto spread over the whole block 208 and thus, abrupt and very localizedobjects within blocks 200 tend to lead to deviations of there-transformed block relative to the original block 200 due toquantization noise. The main advantage of using greater blocks is,however, that the ratio between the number of significant, i.e. non-zero(quantized) transform coefficients on the one hand and the number ofinsignificant transform coefficients on the other hand may be decreasedwithin larger blocks compared to smaller blocks thereby enabling abetter coding efficiency. In other words, frequently, the significanttransform coefficients, i.e. the transform coefficients not quantized tozero, are distributed over the transform block 204 sparsely. Due tothis, in accordance with the embodiments described in more detail below,the positions of the significant transform coefficients is signaledwithin the data stream by way of a significance map. Separatelytherefrom, the values of the significant transform coefficient, i.e.,the transform coefficient levels in case of the transform coefficientsbeing quantized, are transmitted within the data stream.

Accordingly, according to an embodiment of the present application, anapparatus for decoding such a significance map from the data stream orfor decoding the significance map along the corresponding significanttransform coefficient values from the data stream, may be implemented asshown in FIG. 7, and each of the entropy decoders mentioned above,namely decoder 50 and entropy decoder 150, may comprise the apparatusshown in FIG. 7.

The apparatus of FIG. 7 comprises a map/coefficient entropy decoder 250and an associator 252. The map/coefficient entropy decoder 250 isconnected to an input 254 at which syntax elements representing thesignificance map and the significant transform coefficient values enter.As will be described in more detail below, different possibilities existwith respect to the order in which the syntax elements describing thesignificance map on the one hand and the significant transformcoefficient values on the other hand enter map/coefficient entropydecoder 250. The significance map syntax elements may precede thecorresponding levels, or both may be interleaved. However, preliminaryit is assumed that the syntax elements representing the significance mapprecede the values (levels) of the significant transform coefficients sothat the map/coefficient entropy decoder 250 firstly decodes thesignificance map and then the transform coefficient levels of thesignificant transform coefficients.

As map/coefficient entropy decoder 250 sequentially decodes the syntaxelements representing the significance map and the significant transformcoefficient values, the associator 252 is configured to associate thesesequentially decoded syntax elements/values to the positions within thetransform block 256. The scan order in which the associator 252associates the sequentially decoded syntax elements representing thesignificance map and levels of the significant transform coefficients tothe positions of the transform block 256 follows a one-dimensional scanorder among the positions of the transform block 256 which is identicalto the order used at the encoding side to introduce these elements intothe data stream. As will also be outlined in more detail below, the scanorder for the significance map syntax elements may be equal to the orderused for the significant coefficient values, or not.

The map/coefficient entropy decoder 250 may access the information onthe transform block 256 available so far, as generated by the associator252 up to a currently to be decoded syntax element/level, in order toset probability estimation context for entropy decoding the syntaxelement/level currently to be decoded as indicated by a dashed line 258.For example, associator 252 may log the information gathered so far fromthe sequentially associated syntax elements such as the levels itself orthe information as to whether at the respective position a significanttransform coefficient is situated or not or as to whether nothing isknown about the respective position of the transform block 256 whereinthe map/coefficient entropy decoder 250 accesses this memory. The memoryjust mentioned is not shown in FIG. 7 but the reference sign 256 mayalso indicate this memory as the memory or log buffer would be forstoring the preliminary information obtained by associator 252 andentropy decoder 250 so far. Accordingly, FIG. 7 illustrates by crossespositions of significant transform coefficients obtained from thepreviously decoded syntax elements representing the significance map anda “1” shall indicate that the significant transform coefficient level ofthe significant transform coefficient at the respective position hasalready been decoded and is 1. In case of the significance map syntaxelements preceding the significant values in the data stream, a crosswould have been logged into memory 256 at the position of the “1” (thissituation would have represented the whole significance map) beforeentering the “1” upon decoding the respective value.

The following description concentrates on specific embodiments forcoding the transform coefficient blocks or the significance map, whichembodiments are readily transferable to the embodiments described above.In these embodiments, a binary syntax element coded block flag may betransmitted for each transform block, which signals whether thetransform block contains any significant transform coefficient level(i.e., transform coefficients that are non-zero). If this syntax elementindicates that significant transform coefficient levels are present, thesignificance map is coded, i.e. merely then. The significance mapspecifies, as indicated above, which of the transform coefficient levelshave non-zero values. The significance map coding involves a coding ofbinary syntax elements significant_coeff_flag each specifying for arespectively associated coefficient position whether the correspondingtransform coefficient level is not equal to zero. The coding isperformed in a certain scan order which may change during thesignificance map coding dependent on the positions of significantcoefficients identified to be significant so far, as will be describedin more detail below. Further, the significance map coding involves acoding of binary syntax elements last_significant_coeff_flaginterspersed with the sequence of significant_coeff_flag at thepositions thereof, where significant_coeff_flag signals a significantcoefficient. If the significant_coeff_flag bin is equal to one, i.e., ifa non-zero transform coefficient level exists at this scanning position,the further binary syntax element last_significant_coeff_flag is coded.This bin indicates if the current significant transform coefficientlevel is the last significant transform coefficient level inside theblock or if further significant transform coefficient levels follow inscanning order. If last_significant_coeff_flag indicates that no furthersignificant transform coefficients follow, no further syntax elementsare coded for specifying the significance map for the block.Alternatively, the number of significant coefficient positions could besignaled within the data stream in advance of the coding of the sequenceof significant_coeff_flag. In the next step, the values of thesignificant transform coefficient levels are coded. As described above,alternatively, the transmission of the levels could be interleaved withthe transmission of the significance map. The values of significanttransform coefficient levels are coded in a further scanning order forwhich examples are described below. The following three syntax elementsare used. The binary syntax element coeff_abs_greater_one indicates ifthe absolute value of the significant transform coefficient level isgreater than one. If the binary syntax element coeff_abs_greater_oneindicates that the absolute value is greater than one, a further syntaxelement coeff_abs_level_minus_one is sent, which specifies the absolutevalue of the transform coefficient level minus one. Finally, the binarysyntax element coeff_sign_flag, which specifies the sign of thetransform coefficient value, is coded for each significant transformcoefficient level.

The embodiments described below enable to further reduce the bit rateand thus increase the coding efficiency. In order to do so, theseembodiments use a specific approach for context modelling for syntaxelements related to the transform coefficients. In particular, a newcontext model selection for the syntax elements significant_coeff_flag,last_significant_coeff_flag, coeff_abs_greater_one andcoeff_abs_level_minus_one is used. And furthermore, an adaptiveswitching of the scan during the encoding/decoding of the significancemap (specifying the locations of non-zero transform coefficient levels)is described. As to the meaning of the must-mentioned syntax elements,reference is made to the above introductory portion of the presentapplication.

The coding of the significant_coeff_flag and thelast_significant_coeff_flag syntax elements, which specify thesignificance map, is improved by an adaptive scan and a new contextmodelling based on a defined neighborhood of already coded scanpositions. These new concepts result in a more efficient coding ofsignificance maps (i.e., a reduction of the corresponding bit rate), inparticular for large block sizes.

One aspect of the below-outlined embodiments is that the scan order(i.e., the mapping of a block of transform coefficient values onto anordered set (vector) of transform coefficient levels) is adapted duringthe encoding/decoding of a significance map based on the values of thealready encoded/decoded syntax elements for the significance map.

In an embodiment, the scan order is adaptively switched between two ormore predefined scan pattern. In an embodiment, the switching can takeplace only at certain predefined scan positions. In a further embodimentof the invention, the scan order is adaptively switched between twopredefined scan patterns. In an embodiment, the switching between thetwo predefined scan patterns can take place only at certain predefinedscan positions.

The advantage of the switching between scan patterns is a reduced bitrate, which is a result of a smaller number of coded syntax elements. Asan intuitive example and referring to FIG. 6, it is often the case thatsignificant transform coefficient values—in particular for largetransform blocks—are concentrated at one of the block borders 270, 272,because the residual blocks contain mainly horizontal or verticalstructures. With the mostly used zig-zag scan 274, there exists aprobability of about 0.5 that the last diagonal sub-scan of the zig-zagscan in which the last significant coefficient is encountered startsfrom the side at which the significant coefficients are notconcentrated. In that case, a large number of syntax elements fortransform coefficient levels equal to zero have to be coded before thelast non-zero transform coefficient value is reached. This can beavoided if the diagonal sub-scans are started at the side, where thesignificant transform coefficient levels are concentrated.

MORE DETAILS FOR AN EMBODIMENT OF THE INVENTION ARE DESCRIBED BELOW

As mentioned above, also for large block sizes, it is advantageous tokeep the number of context models reasonably small in order to enable afast adaptation of the context models and providing a high codingefficiency. Hence, a particular context should be used for more than onescan position. But the concept of assigning the same context to a numberof successive scan positions, as done for 8×8 blocks in H.264, isusually not suitable, since the significant transform coefficient levelsare usually concentrated in certain areas of a transform blocks (thisconcentration may be a result of certain dominant structures that areusually present in, for example residual blocks). For designing thecontext selection, one could use the above mentioned observation thatsignificant transform coefficient levels are often concentrated incertain areas of a transform block. In the following, concepts aredescribed by which this observation can be exploited.

In one embodiment, a large transform block (e.g., greater than 8×8) ispartitioned into a number of rectangular sub-blocks (e.g., into 16sub-blocks) and each of these sub-blocks is associated with a separatecontext model for coding the significant_coeff_flag andlast_significant_coeff_flag (where different context models are used forthe significant_coeff_flag and last_significant_coeff_flag). Thepartitioning into sub-blocks can be different for thesignificant_coeff_flag and last_significant_coeff_flag. The same contextmodel may be used for all scan positions that are located in aparticular sub-block.

In a further embodiment, a large transform block (e.g., greater than8×8) may be partitioned into a number of rectangular and/ornon-rectangular sub-regions and each of these sub-regions is associatedwith a separate context model for coding the significant_coeff_flagand/or the last_significant_coeff_flag. The partitioning intosub-regions can be different for the significant_coeff_flag andlast_significant_coeff_flag. The same context model is used for all scanpositions that are located in a particular sub-region.

In a further embodiment, the context model for coding thesignificant_coeff_flag and/or the last_significant_coeff_flag isselected based on the already coded symbols in a predefined spatialneighborhood of the current scan position. The predefined neighborhoodcan be different for different scan positions. In an embodiment, thecontext model is selected based on the number of significant transformcoefficient levels in the predefined spatial neighborhood of the currentscan position, where only already coded significance indications arecounted.

MORE DETAILS FOR AN EMBODIMENT OF THE INVENTION ARE DESCRIBED BELOW

As mentioned above, for large block sizes, the conventional contextmodelling encodes a large number of bins (that usually have differentprobabilities) with one single context model for thecoeff_abs_greater_one and coeff_abs_level_minus_one syntax elements. Inorder to avoid this drawback for large block size, large blocks may, inaccordance with an embodiment, be divided into small quadratic orrectangular sub-blocks of a particular size and a separate contextmodelling is applied for each sub-block. In addition, multiple sets ofcontext models may be used, where one of these context model sets isselected for each sub-block based on an analysis of the statistics ofpreviously coded sub-blocks. In an embodiment invention, the number oftransform coefficients greater than 2 (i.e. coeff_abs_level_minus_1>1)in the previously coded sub-block of the same block is used to derivethe context model set for the current sub-block. These enhancements forcontext modelling of the coeff_abs_greater_one andcoeff_abs_level_minus_one syntax elements result in a more efficientcoding of both syntax elements, in particular for large block sizes. Inan embodiment, the block size of a sub-block is 2×2. In anotherembodiment, the block size of a sub-block is 4×4.

In a first step, a block larger than a predefined size may be dividedinto smaller sub-blocks of a particular size. The coding process of theabsolute transform coefficient levels maps the quadratic or rectangularblock of sub-blocks onto an ordered set (vector) of sub-blocks using ascan, where different scans can be used for different blocks. In anembodiment, the sub-blocks are processed using a zig-zag scan; thetransform coefficient levels inside a sub-block are processed in areverse zig-zag scan, i.e. a scan loading from a transform coefficientbelonging to the highest frequency in vertical and horizontal directionto the coefficient relating to the lowest frequency in both directions.In another embodiment of the invention, a reversed zig-zag scan is usedfor coding the sub-blocks and for coding the transform coefficientlevels inside the sub-blocks. In another embodiment of the invention,the same adaptive scan that is used for coding the significance map (seeabove) is used to process the whole block of transform coefficientlevels.

The division of a large transform block into sub-blocks avoids theproblem of using just one context model for most of the bins of a largetransform block. Inside the sub-blocks, the state-of-the-art contextmodelling (as specified in H.264) or a fixed context can be used,depending on the actual size of the sub-blocks. Additionally, thestatistics (in terms of probability modelling) for such sub-blocks aredifferent from the statistics of a transform block with the same size.This property may be exploited by extending the set of context modelsfor the coeff_abs_greater_one and coeff_abs_level_minus_one syntaxelements. Multiple sets of context models can be provided, and for eachsub-block one of these context model sets may be selected based on thestatistics of previously coded sub-block in current transform block orin previously coded transform blocks. In an embodiment of the invention,the selected set of context models is derived based on the statistics ofthe previously coded sub-blocks in the same block. In another embodimentof the invention, the selected set of context models is derived based onthe statistics of the same sub-block of previously coded blocks. In anembodiment, the number of context model sets is set equal to 4, while inanother embodiment, the number of context model sets is set equal to 16.In an embodiment, the statistics that are used for deriving the contextmodel set is the number of absolute transform coefficient levels greaterthan 2 in previously coded sub-blocks. In another embodiment, thestatistics that are used for deriving the context model set is thedifference between the number of significant coefficients and the numberof transform coefficient levels with an absolute value greater than 2.

The coding of the significance map may be performed as outlined below,namely by an adaptive switching of the scan order.

In an embodiment, the scanning order for coding the significance map isadapted by switching between two predefined scan patterns. The switchingbetween the scan patterns can only be done at certain predefined scanpositions. The decision whether the scanning pattern is switched dependson the values of the already coded/decoded significance map syntaxelements. In an embodiment, both predefined scanning patterns specifyscanning patterns with diagonal sub-scans, similar to the scanningpattern of the zig-zag scan. The scan patterns are illustrated in FIG.8. Both scanning patterns 300 and 302 consist of a number of diagonalsub-scans for diagonals from bottom-left to top-right or vice versa. Thescanning of the diagonal sub-scans (not illustrated in the figure) isdone from top-left to bottom-right for both predefined scanningpatterns. But the scanning inside the diagonal sub-scans is different(as illustrated in the figure). For the first scanning pattern 300, thediagonal sub-scans are scanned from bottom-left to top-right (leftillustration of FIG. 8), and for the second scanning pattern 302, thediagonal sub-scans are scanned from top-right to bottom-left (rightillustration of FIG. 8). In an embodiment, the coding of thesignificance map starts with the second scanning pattern. Whilecoding/decoding the syntax elements, the number of significant transformcoefficient values is counted by two counters C₁ and C₂. The firstcounter C₁ counts the number of significant transform coefficients thatare located in the bottom-left part of the transform block; i.e., thiscounter is incremented by one when a significant transform coefficientlevel is coded/decoded for which the horizontal coordinate x inside thetransform block is less than the vertical coordinate y. The secondcounter C₂ counts the number of significant transform coefficients thatare located in the top-right part of the transform block; i.e., thiscounter is incremented by one when a significant transform coefficientlevel is coded/decoded for which the horizontal coordinate x inside thetransform block is greater than the vertical coordinate y. Theadaptation of the counters may be performed by associator 252 in FIG. 7and can be described by the following formulas, where t specifies thescan position index and both counters are initialized with zero:

${C_{1}\left( {t + 1} \right)} = \left\{ {{\begin{matrix}{{1 + {C_{1}(t)}},} & {{x < y}\mspace{40mu}} \\{\mspace{40mu} {{C_{1}(t)},}} & {otherwise}\end{matrix}{C_{2}\left( {t + 1} \right)}} = \left\{ \begin{matrix}{{1 + {C_{2}(t)}},} & {{x > y}\mspace{40mu}} \\{\mspace{40mu} {{C_{2}(t)},}} & {otherwise}\end{matrix} \right.} \right.$

At the end of each diagonal sub-scan, it is decided by the associator252 whether the first or the second of the predefined scanning patterns300, 302 is used for the next diagonal sub-scan. This decision is basedon the values of the counters c₁ and c₂. When the counter for thebottom-left part of the transform block is greater than the counter forthe bottom-left part, the scanning pattern that scans the diagonalsub-scans from bottom-left to top-right is used; otherwise (the counterfor the bottom-left part of the transform block is less than or equal tothe counter for the bottom-left part), the scanning pattern that scansthe diagonal sub-scans from top-right to bottom-left is used. Thisdecision can be expressed by the following formula:

$d_{t + 1} = \left\{ \begin{matrix}{{{top}\mspace{14mu} {right}\mspace{14mu} {to}\mspace{14mu} {left}\mspace{14mu} {bottom}},} & {C_{1} \leq C_{2}} \\{{{left}\mspace{14mu} {bottom}\mspace{14mu} {to}\mspace{14mu} {top}\mspace{14mu} {right}},} & {C_{1} > C_{2}}\end{matrix} \right.$

It should be noted that the above described embodiment of the inventioncan be easily applied to other scanning patterns. As an example, thescanning pattern that is used for field macroblocks in H.264 can also bedecomposed into sub-scans. In a further embodiment, a given butarbitrary scanning pattern is decomposed into sub-scans. For each of thesub-scans, two scanning patterns are defined: one from bottom-left totop-right and one from top-right to bottom-left (as basic scandirection). In addition, two counters are introduced which count thenumber of significant coefficients in a first part (close to thebottom-left border of a transform blocks) and a second part (close tothe top-right border of a transform blocks) inside the sub-scans.Finally, at the end of each sub-scan it is decided (based on the valuesof the counters), whether the next sub-scan is scanned from bottom-leftto top-right or from top-right to bottom-left.

In the following, embodiments for as to how entropy decoder 250 modelsthe contexts, are presented.

In one embodiment, the context modelling for the significant_coeff_flagis done as follows. For 4×4 blocks, the context modelling is done asspecified in H.264. For 8×8 blocks, the transform block is decomposedinto 16 sub-blocks of 2×2 samples, and each of these sub-blocks isassociated with a separate context. Note that this concept can also beextended to larger block sizes, a different number of sub-blocks, andalso non-rectangular sub-regions as described above.

In a further embodiment, the context model selection for largertransform blocks (e.g., for blocks greater than 8×8) is based on thenumber of already coded significant transform coefficients in apredefined neighborhood (inside the transform block). An example for thedefinition of neighborhoods, which corresponds to an embodiment of theinvention, is illustrated in FIG. 9. Crosses with a circle around sameare available neighbors, which are taken into account for the evaluationand crosses with a triangle are neighbors which are evaluated dependingon the current scan position and current scan direction):

-   -   If the current scan position lies in the inside the 2×2 left        corner 304, a separate context model is used for each scan        position (FIG. 9, left illustration).    -   If the current scan position does not lie inside the 2×2 left        corner and is not located on the first row or the first column        of the transform block, then the neighbors illustrated on the        right in FIG. 9 are used for evaluating the number of        significant transform coefficients in the neighborhood of the        current scan position “x” without anything around it.    -   If the current scan position “x” without anything around it        falls into the first row of the transform block, then the        neighbors specified in the right illustration of FIG. 10 are        used.    -   If the current scan position “x” falls in to the first column of        the block, then the neighbors specified in the left illustration        of FIG. 10 are used.

In other words, the decoder 250 may be configured to sequentiallyextract the significance map syntax elements by context-adaptivelyentropy decoding by use of contexts which are individually selected foreach of the significance map syntax elements depending on a number ofpositions at which according to the previously extracted and associatedsignificance map syntax elements significant transform coefficients aresituated, the positions being restricted to ones lying in a neighborhoodof the position (“x” in FIG. 9 right-hand side and FIG. 10 both sides,and any of the marked positions of the left hand side of FIG. 9) withwhich the respective current significance map syntax element isassociated. As shown, the neighborhood of the position with which therespective current syntax element is associated, may merely comprisepositions either directly adjacent to or separated from the positionwith which the respective significance map syntax element is associated,at one position in vertical direction and/or one position in thehorizontal direction at the maximum. Alternatively, merely positionsdirectly adjacent to the respective current syntax element may be takeninto account. Concurrently, the size of the transform coefficient blockmay be equal to or greater than 8×8 positions.

In an embodiment, the context model that is used for coding a particularsignificant_coeff_flag is chosen depending on the number of alreadycoded significant transform coefficient levels in the definedneighborhoods. Here, the number of available context models can besmaller than the possible value for the number of significant transformcoefficient levels in the defined neighborhood. The encoder and decodercan contain a table (or a different mapping mechanism) for mapping thenumber of significant transform coefficient levels in the definedneighborhood onto a context model index.

In a further embodiment, the chosen context model index depends on thenumber of significant transform coefficient levels in the definedneighborhood and on one or more additional parameters as the type of theused neighborhood or the scan position or a quantized value for the scanposition.

For the coding of the last_significant_coeff_flag, a similar contextmodelling as for the significant_coeff_flag can be used. However, theprobability measure for the last_significant_coeff_flag mainly dependson a distance of the current scan position to the top-left corner of thetransform block. In an embodiment, the context model for coding thelast_significant_coeff_flag is chosen based on the scan diagonal onwhich the current scan position lies (i.e., it is chosen based on x+y,where x and y represent the horizontal and vertical location of a scanposition inside the transform block, respectively, in case of the aboveembodiment of FIG. 8, or based on how many sub-scans by between thecurrent sub-scan and the upper left DC position (such as sub-scan indexminus 1)). In an embodiment of the invention, the same context is usedfor different values of x+y. The distance measure i.e. x+y or thesub-scan index is mapped on the set of context models in a certain way(e.g. by quantizing x+y or the sub-san index), where the number ofpossible values for the distance measure is greater than the number ofavailable context models for coding the last_significant_coeff_flag.

In an embodiment, different context modelling schemes are used fordifferent sizes of transform blocks.

In the following the coding of the absolute transform coefficient levelsis described.

In one embodiment, the size of sub-blocks is 2×2 and the contextmodelling inside the sub-blocks is disabled, i.e., one single contextmodel is used for all transform coefficients inside a 2×2 sub-block.Only blocks larger than 2×2 may be affected by the subdivision process.In a further embodiment of this invention, the size of the sub-blocks is4×4 and the context modelling inside the sub-blocks is done as in H.264;only blocks larger than 4×4 are affected by the subdivision process.

As to the scan order, in an embodiment, a zig-zag scan 320 is employedfor scanning the sub-blocks 322 of a transform block 256 i.e. along adirection of substantially increasing frequency, while the transformcoefficients inside a sub-block are scanned in a reverse zig-zag scan326 (FIG. 11). In a further embodiment of the invention, both thesub-blocks 322 and the transform coefficient levels inside thesub-blocks 322 are scanned using a reverse zig-zag scan (like theillustration in FIG. 11, where the arrow 320 is inversed). In anotherembodiment, the same adaptive scan as for coding the significance map isused to process the transform coefficient levels, where the adaptationdecision is the same, so that exactly the same scan is used for both thecoding of the significance map and the coding of the transformcoefficient level values. It should be noted that the scan itself doesusually not depend on the selected statistics or the number of contextmodel sets or on the decision for enabling or disabling the contextmodelling inside the sub-blocks.

Next embodiments for context modelling for the coefficient levels aredescribed.

In an embodiment, the context modelling for a sub-block is similar tothe context modelling for 4×4 blocks in H.264 as has been describedabove. The number of context models that are used for coding thecoeff_abs_greater_one syntax element and the first bin of thecoeff_abs_level_minus_one syntax element is equal to five, with, forexample, using different sets of context models for the two syntaxelements. In a further embodiment, the context modelling inside thesub-blocks is disabled and only one predefined context model is usedinside each sub-block. For both embodiments, the context model set for asub-block 322 is selected among a predefined number of context modelsets. The selection of the context model set for a sub-block 322 isbased on certain statistics of one or more already coded sub-blocks. Inan embodiment, the statistics used for selecting a context model set fora sub-block are taken from one or more already coded sub-blocks in thesame block 256. How the statistics are used to derive the selectedcontext model set is described below. In a further embodiment, thestatistics are taken from the same sub-block in a previously coded blockwith the same block size such as block 40 a and 40 a′ in FIG. 2 b. Inanother embodiment of the invention, the statistics are taken from adefined neighbouring sub-block in the same block, which depends on theselected scan for the sub-blocks. Also, it is important to note that thesource of the statistics should be independent from the scan order andhow the statistics are created to derive the context model set.

In an embodiment, the number of context model sets is equal to four,while in another embodiment, the number of context model sets is equalto 16. Commonly, the number of context model sets is not fixed andshould be adapted in accordance with the selected statistics. In anembodiment, the context model set for a sub-block 322 is derived basedon the number of absolute transform coefficient levels greater than twoin one or more already coded sub-blocks. An index for the context modelset is determined by mapping the number of absolute transformcoefficient levels greater than two in the reference sub-block orreference sub-blocks onto a set of predefined context model indices.This mapping can be implemented by quantizing the number of absolutetransform coefficient levels greater than two or by a predefined table.In a further embodiment, the context model set for a sub-block isderived based on the difference between the number of significanttransform coefficient levels and the number of absolute transformcoefficient levels greater than two in one or more already codedsub-blocks. An index for the context model set is determined by mappingthis difference onto a set of predefined context model indices. Thismapping can be implemented by quantizing the difference between thenumber of significant transform coefficient levels and the number ofabsolute transform coefficient levels greater than two or by apredefined table.

In another embodiment, when the same adaptive scan is used forprocessing the absolute transform coefficient levels and thesignificance map, the partial statistics of the sub-blocks in the sameblocks may be used to derive the context model set for the currentsub-block, or; if available, the statistics of previously codedsub-blocks in previously coded transform blocks may be used. That means,for example, instead of using the absolute number of absolute transformcoefficient levels greater than two in the sub-block(s) for deriving thecontext model, the number of already coded absolute transformcoefficient levels greater than two multiplied by the ratio of thenumber of transform coefficients in the sub-block(s) and the number ofalready coded transform coefficients in the sub-block(s) is used; orinstead of using the difference between the number of significanttransform coefficient levels and the number of absolute transformcoefficient levels greater than two in the sub-block(s), the differencebetween the number of already coded significant transform coefficientlevels and the number of already coded absolute transform coefficientlevels greater than two multiplied by the ratio of the number oftransform coefficients in the sub-block(s) and the number of alreadycoded transform coefficients in the sub-block(s) is used.

For the context modelling inside the sub-blocks, basically the inverseof the state-of-the-art context modelling for H.264 may be employed.That means, when the same adaptive scan is used for processing theabsolute transform coefficient levels and the significance map, thetransform coefficient levels are basically coded in a forward scanorder, instead of a reverse scan order as in H.264. Hence, the contextmodel switching have to be adapted accordingly. According to oneembodiment, the coding of the transform coefficients levels starts witha first context model for coeff_abs_greater_one andcoeff_abs_level_minus_one syntax elements, and it is switched to thenext context model in the set when two coeff_abs_greater_one syntaxelements equal to zero have been coded since the last context modelswitch. In other words, the context selection is dependent on the numberof already coded coeff_abs_greater_one syntax elements greater than zeroin scan order. The number of context models for coeff_abs_greater_oneand for coeff_abs_level_minus_one may be the same as in H.264.

Thus, the above embodiments may be applied to the field of digitalsignal processing and, in particular, to image and video decoders andencoders. In particular, the above embodiments enable a coding of syntaxelements related to transform coefficients in block-based image andvideo codecs, with an improved context modelling for syntax elementsrelated to transform coefficients that are coded with an entropy coderthat employs a probability modelling. In comparison to thestate-of-the-art, an improved coding efficiency is achieved inparticular for large transform blocks.

Although some aspects have been described in the context of anapparatus, it is clear that these aspects also represent a descriptionof the corresponding method, where a block or device corresponds to amethod step or a feature of a method step. Analogously, aspectsdescribed in the context of a method step also represent a descriptionof a corresponding block or item or feature of a correspondingapparatus.

The inventive encoded signal for representing the transform block or thesignificance map, respectively, can be stored on a digital storagemedium or can be transmitted on a transmission medium such as a wirelesstransmission medium or a wired transmission medium such as the Internet.

Depending on certain implementation requirements, embodiments of theinvention can be implemented in hardware or in software. Theimplementation can be performed using a digital storage medium, forexample a floppy disk, a DVD, a Blue-Ray, a CD, a ROM, a PROM, an EPROM,an EEPROM or a FLASH memory, having electronically readable controlsignals stored thereon, which cooperate (or are capable of cooperating)with a programmable computer system such that the respective method isperformed. Therefore, the digital storage medium may be computerreadable.

Some embodiments according to the invention comprise a data carrierhaving electronically readable control signals, which are capable ofcooperating with a programmable computer system, such that one of themethods described herein is performed.

Generally, embodiments of the present invention can be implemented as acomputer program product with a program code, the program code beingoperative for performing one of the methods when the computer programproduct runs on a computer. The program code may for example be storedon a machine readable carrier.

Other embodiments comprise the computer program for performing one ofthe methods described herein, stored on a machine readable carrier.

In other words, an embodiment of the inventive method is, therefore, acomputer program having a program code for performing one of the methodsdescribed herein, when the computer program runs on a computer.

A further embodiment of the inventive methods is, therefore, a datacarrier (or a digital storage medium, or a computer-readable medium)comprising, recorded thereon, the computer program for performing one ofthe methods described herein.

A further embodiment of the inventive method is, therefore, a datastream or a sequence of signals representing the computer program forperforming one of the methods described herein. The data stream or thesequence of signals may for example be configured to be transferred viaa data communication connection, for example via the Internet.

A further embodiment comprises a processing means, for example acomputer, or a programmable logic device, configured to or adapted toperform one of the methods described herein.

A further embodiment comprises a computer having installed thereon thecomputer program for performing one of the methods described herein.

In some embodiments, a programmable logic device (for example a fieldprogrammable gate array) may be used to perform some or all of thefunctionalities of the methods described herein. In some embodiments, afield programmable gate array may cooperate with a microprocessor inorder to perform one of the methods described herein. Generally, themethods are performed by any hardware apparatus.

The above described embodiments are merely illustrative for theprinciples of the present invention. It is understood that modificationsand variations of the arrangements and the details described herein willbe apparent to others skilled in the art. It is the intent, therefore,to be limited only by the scope of the impending patent claims and notby the specific details presented by way of description and explanationof the embodiments herein.

While this invention has been described in terms of several embodiments,there are alterations, permutations, and equivalents which fall withinthe scope of this invention. It should also be noted that there are manyalternative ways of implementing the methods and compositions of thepresent invention. It is therefore intended that the following appendedclaims be interpreted as including all such alterations, permutationsand equivalents as fall within the true spirit and scope of the presentinvention.

What is claimed is:
 1. An apparatus for decoding a transform coefficientblock, the apparatus comprising: a decoder configured to extract, from adata stream, a significance map that indicates positions of significanttransform coefficients within the transform coefficient block, and thesignificant transform coefficients within the transform coefficientblock; and an associator configured to associate the significanttransform coefficients with their respective positions in the transformcoefficient block, wherein transform coefficients within each of aplurality of sub-blocks of the transform coefficient block are scannedin a position sub-scan order, and each sub-block includes informationabout a plurality of pixels, wherein the decoder is configured to use,in extracting the significant transform coefficients, context-adaptiveentropy decoding such that each of the plurality of sub-blocks isentropy decoded using one or more contexts determined for that sub-blockseparately from the other of the plurality of sub-blocks, and wherein,for entropy decoding the significant transform coefficients of asub-block of the plurality of sub-blocks, a selected set of contextsfrom a plurality of sets of contexts is determined based on a specificvalue related to one or more of the significant transform coefficientsof a previously traversed sub-block of the transform coefficient block.2. The apparatus according to claim 1, wherein the number of contexts ofthe plurality of sets of contexts is greater than one, and the decoderis configured to, in context-adapted entropy decoding the significanttransform coefficients within one of the plurality of sub-blocks usingthe selected set of contexts, assign the selected set of contexts to thepositions within the respective sub-block.
 3. The apparatus according toclaim 1, wherein the plurality of sub-blocks of the transformcoefficient block are scanned in a sub-block scan order.
 4. Theapparatus according to claim 1, wherein the apparatus is at least aportion of a programmable logic device, programmable gate array,microprocessor, computer or hardware apparatus.
 5. The apparatusaccording to claim 1, wherein the data stream comprises at least aportion associated with color samples.
 6. The apparatus according toclaim 1, wherein the data stream comprises at least a portion associatedwith depth values related to a depth map.
 7. The apparatus according toclaim 1, wherein the position sub-scan order is adaptively selected froma plurality of scan orders including a diagonal scan order.
 8. Anapparatus for encoding a transform coefficient block, the apparatuscomprising: an encoder configured to encode, into a data stream, asignificance map that indicates positions of significant transformcoefficients within the transform coefficient block, and the significanttransform coefficients within the transform coefficient block, whereintransform coefficients within each of a plurality of sub-blocks of thetransform coefficient block are scanned in a position sub-scan order,and each sub-block includes information about a plurality of pixels,wherein the encoder is further configured to use, in encoding thesignificant transform coefficients, context-adaptive entropy encodingsuch that each of the plurality of sub-blocks is entropy encoded usingone or more contexts determined for that sub-block separately from theother of the plurality of sub-blocks, wherein, for entropy encoding thesignificant transform coefficients of a sub-block of the plurality ofsub-blocks, a selected set of contexts from a plurality of sets ofcontexts is determined based on a specific value related to one or moreof the significant transform coefficients of a previously traversedsub-block of the transform coefficient block.
 9. The apparatus accordingto claim 8, wherein the number of contexts of the plurality of sets ofcontexts is greater than one, and the encoder is configured to, incontext-adapted entropy encoding the significant transform coefficientswithin one of the plurality of sub-blocks using the selected set ofcontexts, assign the selected set of contexts to the positions withinthe respective sub-block.
 10. The apparatus according to claim 8,wherein the plurality of sub-blocks of the transform coefficient blockare scanned in a sub-block scan order.
 11. The apparatus according toclaim 8, wherein the apparatus is at least a portion of a programmablelogic device, programmable gate array, microprocessor, computer orhardware apparatus.
 12. The apparatus according to claim 8, wherein thedata stream comprises at least a portion associated with color samples.13. The apparatus according to claim 8, wherein the data streamcomprises at least a portion associated with depth values related to adepth map.
 14. The apparatus according to claim 8, wherein the positionsub-scan order is adaptively selected from a plurality of scan ordersincluding a diagonal scan order.
 15. The apparatus according to claim10, wherein the sub-block scan order leads zigzag-wise from a sub-blockincluding a position of a lowest frequency in a vertical direction andin a horizontal direction to a sub-block including a position of ahighest frequency in both the vertical and horizontal directions.
 16. Anon-transitory computer-readable medium for storing data associated witha transform coefficient block, comprising: a data stream stored in thenon-transitory computer-readable medium, the data stream comprising anencoded significance map indicating positions of significant transformcoefficients within a transform coefficient block, and the significanttransform coefficients within the transform coefficient block, whereinthe significant transform coefficients are encoded into the data streamby context-adaptive entropy encoding such that each of a plurality ofsub-blocks of the transform coefficient block is entropy coded using oneor more contexts determined for that sub-block separately from the otherof the plurality of sub-blocks, each of the plurality of sub-blocksincluding information about a plurality of pixels, and transformcoefficients within each of the plurality of sub-blocks of the transformcoefficient block are scanned in a position sub-scan order, wherein thesignificant transform coefficients of a sub-block of the plurality ofsub-blocks are context-adapted entropy encoded using a selected set ofcontexts from a plurality of sets of contexts, the selected set ofcontexts being determined based on a specific value related to one ormore of the significant transform coefficients of a previously traversedsub-block of the transform coefficient block.
 17. The non-transitorycomputer-readable medium according to claim 16, wherein the number ofcontexts of the plurality of sets of contexts is greater than one, andwherein, in context-adapted entropy encoding the significant transformcoefficients within one of the plurality of sub-blocks using theselected set of contexts for the respective sub-block, the selected setof contexts are assigned to the positions within the respectivesub-block.
 18. The non-transitory computer-readable medium according toclaim 16, wherein the plurality of sub-blocks of the transformcoefficient block are scanned in a sub-block scan order, and thesub-block scan order leads zigzag-wise from a sub-block including aposition of a lowest frequency in a vertical direction and in ahorizontal direction to a sub-block including a position of a highestfrequency in both the vertical and horizontal directions.
 19. Thenon-transitory computer-readable medium according to claim 16, whereinthe data stream comprises at least a portion associated with colorsamples.
 20. The non-transitory computer-readable medium according toclaim 16, wherein the data stream comprises at least a portionassociated with depth values related to a depth map.
 21. Thenon-transitory computer-readable medium according to claim 16, whereinin accordance with the position sub-scan order, each of the plurality ofsub-blocks is scanned from a position within the respective sub-blockcorresponding to a highest frequency in both vertical and horizontaldirections to a position of the respective sub-block corresponding to alowest frequency in both the vertical and horizontal directions.
 22. Thenon-transitory computer-readable medium according to claim 16, whereinthe data stream is processed by an apparatus, which is at least aportion of a programmable logic device, programmable gate array,microprocessor, computer or hardware apparatus.
 23. The non-transitorycomputer-readable medium according to claim 16, wherein the positionsub-scan order is adaptively selected from a plurality of scan ordersincluding a diagonal scan order.
 24. The non-transitory computerreadable digital storage medium according to claim 16, wherein themedium is a DVD, Blue-Ray, CD, ROM, PROM, EPROM, EEPROM or FLASH memory.25. A method for decoding a transform coefficient block, the methodcomprising: extracting, from a data stream, a significance map thatindicates positions of significant transform coefficients within thetransform coefficient block, and the significant transform coefficientswithin the transform coefficient block; associating the significanttransform coefficients with their respective positions in the transformcoefficient block, wherein transform coefficients within each of aplurality of sub-blocks of the transform coefficient block are scannedin a position sub-scan order, and each sub-block includes informationabout a plurality of pixels; and using, in extracting the significanttransform coefficients, context-adaptive entropy decoding such that eachof the plurality of sub-blocks is entropy decoded using one or morecontexts determined for that sub-block separately from the other of theplurality of sub-blocks, wherein the context-adaptive entropy decodingof the significant transform coefficients of a sub-block of theplurality of sub-blocks includes determining a selected set of contextsfrom a plurality of sets of contexts based on a specific value relatedto one or more of the significant transform coefficients of a previouslytraversed sub-block of the transform coefficient block.
 26. The methodaccording to claim 25, wherein the number of contexts of the pluralityof sets of contexts is greater than one, and the context-adapted entropydecoding the significant transform coefficients within one of theplurality of sub-blocks using the selected set of contexts includesassigning the selected set of contexts to the positions within therespective sub-block.
 27. The method according to claim 25, wherein theplurality of sub-blocks of the transform coefficient block are scannedin a sub-block scan order.
 28. The method according to claim 25, whereinthe data stream comprises at least a portion associated with colorsamples.
 29. The method according to claim 25, wherein the data streamcomprises at least a portion associated with depth values related to adepth map.
 30. The method according to claim 25, wherein the positionsub-scan order is adaptively selected from a plurality of scan ordersincluding a diagonal scan order.